With continued deployments of higher speed data networks and new devices that leverage such data networks, content providers are extending beyond static content to offer streaming content over these data networks. Streaming content includes live streaming whereby content is distributed to end users as it is being published by a content provider. Live streaming includes audio and video streams of live events such as sporting events, news, etc. Streaming content can also include previously broadcast or recorded events such as television programming, movies, music, and other such multimedia content that is available on-demand to end users.
Server-side archiving is a process of recording a live stream so that the stream can subsequently be accessed on-demand. Server-side archiving provides convenience to the content provider as the content provider publishes the content stream once for the dual purpose of making the content stream available for live distribution to end users and for recoding the content stream for subsequent on-demand access.
FIG. 1 illustrates a prior art process for performing server-side archiving. The figure includes a content provider 110, streaming server 120, storage server 130, and end user 140. The content provider 110 publishes a content stream 115 to the streaming server 120. The streaming server 120 distributes the stream 115 to multiple end users including the end user 140. End users include any user with a network connected device such as smartphone, tablet, laptop, or other computing machine.
While the content is being streamed to the streaming server 120, the streaming server 120 performs server-side archiving. Specifically, the streaming server 120 records the stream 115 as a file on the storage of the storage server 130. Once the live stream is complete, end users can access the stream 115 on-demand by requesting that the storage server 130 stream the content that was recorded to the file. The storage server 130 and the streaming server 120 may be separate servers or a single server.
In many instances, content providers do not host or distribute their own streams because of the costs associated with doing so. Specifically, a content provider will need sufficient bandwidth to stream live and on-demand content to hundreds or thousands of end users. Additionally, the content provider will need sufficient storage resources to store the on-demand content. Many content providers therefore turn to a content streaming system for the live and on-demand distribution of their content streams.
The content streaming system provides an infrastructure with sufficient bandwidth, storage, and other resources to distribute large amounts of content to large quantities of end users. However, within the content streaming system, server-side archiving becomes a complex issue.
FIG. 2 illustrates one prior art approach to server-side archiving within a content streaming system 205. As shown, the content streaming system 205 includes an ingest cluster 210, storage server 240, and a set of edge servers 250. The ingest cluster 210 includes a load balancer 215 and streaming servers 220 and 230. Each streaming server 220 and 230 includes an internal storage 235.
In FIG. 2, the load balancer 215 distributes the content providers 260 and 270 across the ingest cluster 210 such that the content provider 260 publishes content to the streaming server 220 and the content provider 270 publishes content to the streaming server 230. The published content is distributed from the streaming servers 220 and 230 to multiple end users using the set of edge servers 250. Additionally, the published content is recorded to a file that is stored on the internal storage 235 of the streaming server to which the content is being streamed.
Once a content provider stops publishing content to a particular streaming server, the particular streaming server transfers the recorded file from the internal storage 235 to the storage server 240. The set of edge servers 250 can then distribute the content on-demand to end users by accessing the recorded file on the storage server 240.
A problem occurs in the content streaming system 205 when a content provider is disconnected from the streaming server and the content provider reconnects to resume a live stream. As shown in FIG. 3, when the content provider 310 initially connects to the content streaming system 305, the load balancer 315 forwards the content provider 310 to a first streaming server 320 of the ingest cluster 330. The content provider 310 begins publishing a live stream to the first streaming server 320 and the first streaming server 320 records the live stream to a file on its local storage 335.
During the live stream, the content provider may be disconnected from the first streaming server 320 due to a network error (e.g., network congestion) or because the content provider 310 choose to interrupt, pause, or stop the live stream. When the content provider 310 reconnects to the ingest cluster 330 in order to resume the live stream, the load balancer 315 may forward the content provider 310 to a second streaming server 325 of the ingest cluster 330. The content provider 310 resumes publishing the remainder of the live stream to the second streaming server 325 and the second streaming server 325 records the latter portion of the live stream to a file on its local storage 345. As a result, the live stream becomes segmented into two separate files.
When the files are transferred to the storage server 340, one file may overwrite the other file such that the available on-demand content stream is incomplete. Alternatively, the storage server 340 may retain both files. However, when an end user request for the content stream arrives at the storage server 340, the request will specify a single file and not each of the at least two segmented files. This is because the content provider provides the end user with the link to the on-demand content. The content provider assumes that its streamed content was recorded and stored as a single file on the storage server 340 and the content provider is therefore unaware of the file segmentation.
To remedy the file segmentation issue, the storage server 340 can perform transcoding to combine the segmented files into one file. However, the resources needed to perform such transcoding can degrade the overall system performance. Transcoding is a resource intensive process that takes several minutes or hours for the storage server 340 to complete. When transcoding, many of the storage server's resources are used to perform the transcoding instead of servicing on-demand requests. Moreover, transcoding may not be possible when the content provider publishes a content stream that is encoded using an encoder that is not supported by or unknown to the storage server 340.
Accordingly, there is a need for a content streaming system that performs live distribution of content and server-side archiving for multiple content providers and end users. There is a need for such a system to perform server-side archiving in a manner that does not degrade overall system performance and does not subject the system to the issues of file segmentation and transcoding. Furthermore, there is a need for scalability within the content streaming system such that the system can perform live distribution and server-side archiving for any number of content providers, end users, live streams, and on-demand streams.